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ABSTRACT 

Goal 5 of "The Illinois Commitment," strategic plan for 
higher education in Illinois, stipulates that by 2004, all academic programs 
will systematically assess student learning and use assessment results to 
improve programs. This paper provides the history and context for the 
systematic assessment of student learning, including end-of-program 
evaluation. It also provides the definition of "assessment of student 
learning" used by the Illinois Board of Higher Education, and it clarifies 
the connections between assessment for student learning and other quality 
assurance elements established by the Board. The paper highlights the 
evidence needed for the assurance of quality, improvement, and 
accountability, and it proposes next steps and timelines for collaboration 
and the implementation of Goal 5 requirements. Key elements for "The Illinois 
Commitment" assessment of student learning are: (1) a statement of program 

goals and intended student learning outcomes developed by each program's 
faculty to reflect the uniqueness of that program; (2) systematic assessment 
of student learning using multiple qualitative and quantitative measures; (3) 
feedback from key stakeholders ; (4) evidence of a formal and effective 

feedback/improvement mechanism; (5) monitoring of findings and 
recommendations by the institution at least once a year; and (6) assessment 
and improvement results submitted to the Illinois Board of Higher Education 
as part of an institution's normal schedule. Statewide conversations and 
workshops are planned to promote discussion about the Goal 5 objectives. 

( SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. . 



HE 035 312 



t"' 

oo 

o\ 

vo 

Q 

W 



Putting Students First: 

Assessing Mastery of Student Learning 

"The Illinois Commitment'Goal 5 Requirement: 
Assessment of Student Learning and Improving Program 

Quality 



2002 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
, / CENTER (ERIC) 

This document has been reproduced as 
received from the person or organization 
originating it. 



□ Minor changes have been made to improve 
reproduction quality 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 

C . Lor -ton 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



BEST COPY AVAILABLE 




2 






Putting Students First: 
Assessing Mastery of Student Learning 



THE ILLINOIS COMMITMENT GOAL 5 REQUIREMENT: 
ASSESSMENT OF STUDENT LEARNING AND 
IMPROVING PROGRAM QUALITY 



Introduction 

The Illinois Board of Higher Education (IBHE) committed to re-evaluate and redesign its 
quality assurance processes for program approval and program review in October 1998, for the 
purpose of making them consistent with the values and priorities of a strategic plan that was, then, 
under development. The Board adopted that plan, The Illinois Commitment, in February 1999. 
Goal 5 of The Illinois Commitment focuses on the centrality of quality in Illinois postsecondary 
education: “Illinois colleges and universities will hold students to even higher expectations for 
learning and will be accountable for the quality of academic programs and the assessment of 
learning.” To further quality assurance, Goal 5 stipulates that, “By 2004, all academic 
programs will systematically assess student learning and use assessment results to improve 
programs.” 

The Board of Higher Education defines The Illinois Commitment requirement for 
“assessment of student learning in all programs” as the systematic (at different points throughout 
the program, including end-of-program evaluation) determination of what students know and are 
able to do as a result of completing a unique program of study. This assessment should include 
multiple qualitative and quantitative measures of student learning. These assessment results must 
then be used to improve the quality of curriculum, teaching, and student learning. 

The Board believes that general education is the foundation of all academic programs 
and, thus, is of critical importance to the quality of each program, as well as a student’s ability to 
apply content to the major. In addition, the Board believes that it is important to identify 
weaknesses in the general education curriculum and student performance before students 
complete their program of study. General education provides a broad foundation for all 
subsequent in-depth learning. It introduces students to a diversify of disciplinary content and a 
range of methodological and ontological approaches to organizing knowledge. The 
multidisciplinary skills and perspectives learned in general education enrich the deeper, more 
focused study in the major. 

The purposes of this paper are: to provide the history and context for the systematic 
assessment of student learning, including end-of-program evaluation; to provide the Board’s 
definition of “assessment of student learning;” to clarify the connections between assessment of 
student learning and other IBHE quality assurance elements; to highlight the evidence needed for 
the assurance of quality, improvement, and accountability; and to propose next steps and 
timelines for collaboration and implementation of the Goal 5 requirement. 

Illinois Board of Higher Education Quality Assurance Elements 

Quality assurance is not new to Illinois institutions of higher education. Institutions 
regularly undergo quality review and evaluation from regional accrediting associations, and 
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special and professional accrediting bodies periodically review and accredit or deny accreditation 
for particular academic programs. In addition, some institutions conduct their own internal 
quality reviews, some using external reviewers. 

Historically, IBHE had two quality assurance requirements in place to determine and 
monitor quality and report on performance: Academic Program Approval and Academic 

Program Review. In the Academic Program Approval process, the Board required proposed 
degree programs to undergo a rigorous analysis to meet specific Board criteria in order to gain 
approval prior to implementation. In Academic Program Review, the Board required 
institutions to review all certificate and degree programs every eight years on an IBHE-mandated 
schedule to monitor quality, identify needed improvements, or make recommendations to boards 
of trustees for closure. 

In 1999, the Board adopted The Illinois Commitment, a framework for focusing on the 
priority needs of the state and its residents. As a means of assuring quality, progress and public 
accountability for this set of strategic priorities, the Board required a third quality assurance and 
accountability process— an annual progress report on the six goals, the Institutional Results 
Reports, beginning in August 1999. Included in The Illinois Commitment were two other quality 
and accountability requirements: Assessment of Student Learning and Statewide Performance 
Measures. 

In 1998, the Board committed to re-evaluate and redesign its quality assurance processes 
and its reporting requirements. In the interim, Board staff worked with the Chief Academic 
Officers of the public colleges and universities and a Redesign Working Group to remove 
redundancies and sharpen focus in the processes and reporting. Academic Program Approval and 
Academic Program Review were redesigned to focus on outcomes and include the requirement of 
Assessment of Student Learning as criteria for approval and review. In addition, Board staff has 
worked with the Chief Academic Officers of the public colleges and universities to develop 
guidelines, key elements, and next steps for The Illinois Commitment requirement for assessment 
of student learning. Another group of institutional representatives, some of whom are Chief 
Academic Officers, currently is working with Board staff to develop Statewide Performance 
Measures for the six goals of The Illinois Commitment. 

By 2004, the redesigned IBHE quality assurance processes will include four integrated 
elements: Academic Program Approval, Assessment of Student Learning, Academic Program 
Review and Improvement, and Institutional Results Reports. An additional element of the 
Board’s quality and accountability processes is Statewide Performance Measures, currently under 
development with institutional representatives. 

Redesigned Program Approval shifted emphasis from inputs to outcomes and removed 
redundancies in processes required for approval. Recent minor modifications to the application 
form will: sharpen the focus of questions; explicitly integrate the requirements of Assessment of 
Student Learning, including requiring evidence of a commitment to a monitoring and feedback 
mechanism for improving curriculum, instruction, and learning; require inclusion of a statement 
of expected student learning outcomes; and require a description of the process and elements by 
which student learning will be assessed in order to gain approval. 

Redesigned Academic Program Review decentralizes timetables, priorities and 
processes to the institutions, but requires institutions to prioritize programs for review that may 
have quality or viability issues; implement Assessment of Student Learning in all academic 
programs; provide evidence of findings and recommendations; and submit evidence of a 



formalized and effective feedback mechanism. The Board will continue the requirement that all 
current academic programs be reviewed at least once every eight years; however, in the 
redesigned Program Review process, institutions determine their own schedule for reviewing 
programs and may use the reviews from specialized accreditors or other internal or external 
review processes, rather than being required to conduct a redundant review for IBHE. The 
redesigned Program Review shifts the emphasis to student learning outcomes and to a feedback 
mechanism for on-going faculty engagement to improve curriculum, assessment, teaching, and 
learning. 

Assessment of Student Learning is the systematic (at different points throughout the 
program, including end-of-program evaluation) determination of what students know and are able 
to do as a result of completing a unique program of study. This assessment should include 
multiple qualitative and quantitative measures of student learning. These assessment results must 
then be used to improve the quality of curriculum, teaching, and student learning. Assessment of 
Student Learning is the central integrating quality assurance and improvement element between 
Academic Program Approval and Academic Program Review and Improvement. 

Institutional Results Reports, in addition to containing the progress reports on the six 
goals, will be the vehicle to which a summary of findings and recommendations from the 
Program Review and Improvement and evidence regarding Assessment of Student Learning (in 
the reviewed program) will be appended. The Results Report also will serve as the vehicle for an 
institution’s reporting to the Board in its Statewide Performance Measures. 

Statewide Performance Measures, currently under development with institutional 
leaders, will measure an institution’s progress toward the six goals of The Illinois Commitment 
These performance measures include three types of indicators directly related to The Illinois 
Commitment: 

• State-level indicators - pertaining to the performance of Illinois’ system of higher 
education as a whole; 

• “Common” institutional indicators - related to the statewide goals for higher education 
enumerated in The Illinois Commitment. 

• Mission-specific indicators - pertaining to each institution’s unique role and mission 
within the overall context of The Illinois Commitment. 

Institutions will select a limited number of “mission specific indicators” and identify these 
indicators (no data required in 2002) as a part of their 2002 Results Reports. 

Inputs Assessment — An Historical Focus 

The assessment of quality in higher education has traditionally focused on inputs: 
faculty, curriculum, incoming students, and the fiscal, administrative, and physical plant 
infrastructures of institutions. These indicators continue to serve as hallmarks of quality with 
reviews occurring at the national, state, and institutional levels. Regional accrediting bodies play 
a powerful role in defining, evaluating, and validating the integrity of all institutions of higher 
education. Accreditation by professional associations is another long-standing approach to 
assessing program quality. Many academic programs undergo rigorous inspection and regular 
oversight by such specialized accreditors as the Accreditation Board for Engineering and 
Technology (ABET), the American Psychological Association (APA), and the National Council 
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for Accreditation of Teacher Education (NCATE). While not all academic programs have a 
formal accreditation process, many professional societies, such as the American Historical 
Association, regularly devote portions of their annual meetings to exploring quality in their 
respective disciplines. 

Academic quality is further monitored by a variety of professional boards and societies 
that oversee the licensure and certification of professionals in their fields. State governments 
authorize many of these boards. Various state agencies also conduct institutional and program 
reviews and are responsible for oversight of programs at many levels. Many institutions also 
assess the activities of faculty and students in an academic unit through their own regular 
program review, often employing faculty consultants external to the unit and/or the state in the 
program review process. 

Outcomes Assessment — A Comprehensive Emphasis 

In recent years, external stakeholders have wanted to know what the outcomes of learning 
are, and they wanted to be assured that institutions are committed to continuous quality 
improvement. While there has been a strong tradition of commitment to outcomes assessment in 
many fields, there also has been a call for the development and measurement of learning and 
other outcomes in all programs, including general education. Responding to internal imperatives 
and societal pressure, states and institutions have developed increasingly more focused means of 
identifying and measuring quality, including measuring student-learning outcomes. It is 
important that these measures not be reduced to a single instrument such as a standardized test. 
Rather, meaningful assessment of student learning outcomes requires multiple measures that 
include both qualitative and quantitative approaches at different points throughout the program, 
including end-of-program assessment. 

In general, outcomes assessment methods focus on processes and products, ensuring that 
assessment results are implemented to yield improvements in curriculum, instruction, and student 
learning. Key questions in outcomes-based assessment of student learning include: 

• What should students know and be able to do as a result of having completed a program 
of study? 

• Did the students achieve what they were reasonably expected to achieve? 

• How do we know? 

• If the desired outcomes were not achieved, how is assessment information used to 
improve curriculum, teaching, and learning? 

• What is the evidence that monitoring feedback and adjustments actually improved the 
curriculum, instruction, and student learning? 

Key Concepts in Assessing Student Learning and Improving Program Quality 

A recent American Association of Higher Education (AAHE) document entitled 
Principles of Good Practice for Assessing Student Learning, includes the following 
characteristics of a good assessment process: The educational enterprise subscribes to a 

statement of values; a definition of learning has been developed (across time, multidimensional, 
integrated); the purposes of the educational experience are clearly stated; there is a recognition 



ERIC 



4 



6 



that outcomes are important, but so also is the process that got the student there; and quality is 
defined by the whole academic community. 

In recent years, the Higher Learning Commission/North Central Association (NCA), 
along with other regional accrediting bodies, has turned to formalizing this assessment of student 
learning outcomes and has identified the assessment process as a necessary element of quality. 
The North Central Association requires that every accredited institution implement a 
comprehensive process to assess student academic achievement. Key components of the NCA 
process include: “identifying learning expectations, determining the outcomes of assessing 
student learning across academic programs, and using assessment results to improve student 
learning.” 

For example, NCA differentiates between direct and indirect measures of student 
achievement. Direct measures include evaluation of capstone experiences, internships, and 
student portfolios; performance on locally developed student performance instruments, 
standardized tests, or on professional licensure and certification exams; performance on affective 
measures such as sensitivity to cultural and global diversity, evidence of good citizenship (values, 
ethics, engagement); and effective demonstration of the skill of learning as an outcome. Indirect 
measures include surveys of student satisfaction, alumni job placement/satisfaction, employment, 
and employer satisfaction as outcomes. 

Assessment of general education is key to evaluating the outcomes of the baccalaureate 
degree. To date, most assessment efforts have focused on specific units of knowledge and 
content or skills assessment; however, accrediting bodies have shown greater interest in overall 
general education assessment as they have turned their attention to student learning outcomes. 
Overall, outcomes assessment is, to a large extent, a work-in-progress, but one that will likely 
merit closer scrutiny and further refinement. 

Recommended IBHE Guidelines for Goal 5: Assessment of Student Learning 
and Program Improvement 

The following guidelines should inform the development of student learning assessment 
and program improvement: 

1 . Assessment plans and quality processes should be faculty, program, and campus-driven. 

2. Assessment plans and program approval and review processes should build on existing 
activities, i.e., integrate and expand on existing assessment activities. 

3. Assessment activities should focus on the measurement and improvement of student 
learning outcomes, including multiple qualitative and quantitative assessments, as 
appropriate to the discipline. 

4. Assessment of mastery and quality should not be a one-time event, but rather, a 
continuing process that monitors and self-regulates the educational enterprise to ensure 
that quality is continually enhanced. 

Key Elements for The Illinois Commitment Assessment of Student Learning 

The Illinois Board of Higher Education recommends that all program assessments 
include the following six elements: 
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1 . A statement of program goals and intended student learning outcomes developed by each 
program’s faculty that reflects uniqueness of that program. 

2. Systematic (at different points throughout the program, including end-of-program 
evaluation) assessment of student learning that uses multiple qualitative and quantitative 
measures and reflects the uniqueness of academic programs and disciplines (e.g., 
evaluation of capstone experiences, internships, portfolios, and other types of 
performance measurements; performance on standardized, locally-developed, or 
professional licensure and certification exams). 

3. Feedback gathered from key stakeholders — current students, alumni, and employers of 
graduates, graduate schools, etc., (e.g., surveys of student and alumni satisfaction; alumni 
job placement information; employer satisfaction). 

4. Evidence of a formal and effective feedback/improvement mechanism, i.e., that the 
program faculty are engaged in a regular assessment and review process, and that the 
assessment of student learning and stakeholder feedback are used to improve curriculum, 
instruction, and learning. 

5. Findings and recommendations for improvement are monitored by the institution 
for results at least yearly. 

6. Assessment and improvement results are submitted to IBHE as part of an institution’s 
normal schedule for reporting Program Review findings and recommendations, which are 
appended to the Institutional Results Report. 

Proposed Next Steps and Timelines 

General Education and Discipline-Specific Assessment Workshops 

In addition to its broader purpose, this paper is intended to serve as the basis for a 
state-level workshop and discipline-specific and general education discussions about student 
learning assessment involving academic leaders from two- and four-year institutions across the 
state. Participants in the initial state-level workshop will include institutional teams comprised of 
deans, department chairs, faculty members, assessment coordinators, and others as appropriate to 
the institution. In addition to reviewing the IBHE assessment expectations, these conversations 
are meant to clarify and solidify the understandings regarding the assessment requirement and its 
critical connection to the Program Review and Improvement, Institutional Results Reports, and 
Program Approval processes. 

At two summer workshops (June 18 and July 24, 2002), academic leaders will explore 
the principles and practices outlined in this paper to determine how they may best be adapted by 
individual programs to implement assessment of student learning. It will be especially important 
to the initial statewide discussion that departmental/discipline leadership be engaged, since these 
individuals will lead the implementation of student learning assessment at the program level, for 
the purpose of improving curriculum, teaching, and student learning outcomes. 

These summer workshops will feature Illinois “experts” on assessment, including 
presentations by beginning, developing, and mature assessment programs. The Board will invite 
Illinois institutions to submit their own best-practice assessment and improvement models in 
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Arts, Sciences, Humanities, Professional, and Occupational programs. Discipline-specific and 
various maturity-level models (beginning, implementation, and established) will be featured at 
the workshop. 

These statewide conversations, together with feedback from Illinois academic leadership, 
will result in specific outcomes: (1) holding regional or discipline-specific workshops to further 
operationalize best practices in assessing student learning outcomes and program improvement 
models; (2) providing incentives for institutions to hold similar institutional-level follow-up 
workshops to disseminate the findings of these statewide conversations; and (3) submitting brief 
implementation status reports on assessment activities appended to the 2002, 2003, and 2004 
Results Report. 
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Timeline 



Illinois Board of Higher Education staff has established the following timeline for 
implementation of these recommended principles and guidelines. 



December -February 2002 


Share paper with Academic Leadership of public 
colleges and universities for comment and 
refinement. 


March-April 2002 


Disseminate revised paper to all Chief Academic 
Officers and an RFP to solicit best practice models 
of student learning assessment and program 
improvement. 


June, July 2002 


Convene a statewide workshop for institutional 
teams for the purposes of disseminating IBHE 
overview of assessment of student learning 
guidelines, expectations, and clarifying 

connections to other IBHE quality assurance 
requirements; and presenting best practice 
assessment and improvement models from Illinois 
institutions, with breakout sessions devoted to 
general education and various academic, 
professional and occupational disciplines, 
including beginning, developing, and mature 
assessment and improvement models, as well as 
institutional team planning. 


Fall 2002 


Convene regional, on-campus, and follow-up 
assessment workshops. 


August 2002; August 2003; 
August 2004 


Brief status report (appended to the Institutional 
Results Report) on the development and 
implementation of assessment plans for general 
education and each academic program. 


Academic Year 2004 -2005 


Implement The Illinois Commitment requirement 
that, “all academic programs will systematically 
assess student learning and use assessment to 
improve programs” using the IBHE definition, 
guidelines, and key elements. 
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